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Preface 



The methods and assessment instruments that comprise this Handbook 
of Ideas for Evaluating Workplace Literacy Programs were developed over 
a period of several years. They arose out of a perceived need to have 
something other than a standardized reading test for evaluating learner 
gains in workplace programs. Standardized tests r?re inappropriate in 
this context for two reasons: they test general reaciiig skills whereas 
workplace instruction is often job-specific, and they are unlikely to show 
learner gain in the short time period available for most workplace 
programs. Thus, in order to demonstrate what workers have learned in 
brief and targeted training, there is a need for job- and instruction- 
related measures of achievement— many of them custom-designed for 
each particular workplace. Therefore this Handbook consists, not only of 
actual instruments ready to use, but also of instructions and templates for 
developing instruments for particular workplace situations. 

Workplace literacy programs typically have a wide variety of goals. Most 
are designed to improve learner abilities, but many hope also to help 
learners make changes in their life-styles in relation to literacy and 
education. This often includes increasing the practice of literacy at home 
and on the job, as well as helping learners build on-going education into 
their plans for the future. Still other workplace literacy programs aim for 
direct impact upon participation in quality assurance groups, quality 
monitoring and other productivity- related activities. The broad definition 
of literacy incorporated into this assessment model allows the tracking of 
program impacts in each of these areas. 

This evaluation model, with some instruments that can be used by any 
program and others that need adapting to each particular program, has 
passed through various stages of development. Under the authors' 
supervision, it has been used at about 10 workplace sites during the last 
three years and feedback from the workplace educators using it has 
produced a number of revisions to both the overall structure and many of 
the details of procedure and wording. A version of the model was 
published earlier 1 , and this Handbook can be seen as an up-date of the 
Appendices of that earlier publication. 

As just mentioned, this revised set of instruments owes much to its users 
at various workplaces and we would like to acknowledge particularly the 
assistance of Denise Henard, David Mathes, Daniel O'Connell and Wilma 
Sheffer. 



Mikulecky, Larry & Lloyd, Paul (1992). Evaluating the Impact of workplace literacy 
programs: Results and Instruments from the NCAL Workplace Literacy Impact 
Project. Philadelphia, PA: National Center on Adult Literacy. (ERIC Document 
Reproduction Service No. ED 348 579.) 
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Introduction 



This handbook provides guidelines for designing evaluation materials and for conducting the 
evaluation of your workplace training program. The purpose of a program evaluation is to find out 
whether your program is working effectively in achieving its goals. Program goals may include 
making employees better at their jobs, increasing employees' general education level, or raising 
employee morale and self-esteem. 

Whatever those goals may be, a program evaluation aims to find out whether the 
funding allocated to your training program is money well spent* 

Because of this commitment of scarce resources, it is worth taking a little trouble to find out what 
your training program has achieved. In order to do that, you need to evaluate the things this 
program was aiming to teach. That means some custom-designing of the evaluation instruments, 
so that the measures you use are able to show the improvements made by your program. 

For example, if your training program concentrates on teaching Statistical Process Control, then 
the best ways to measure the effectiveness of the program are by testing employees with scenario 
situations based on SPC skills, and by asking their supervisors to rate their ability to carry out SPC 
on the job. For this class, it does not make sense to use, for example, a standardized test—- which 
tests general ability— because it does not necessarily relate to the ideas taught in the class, and 
employees are likely to show minimal or no gains on such a test. 

Mention of "gains" brings us to another important point in program evaluation: what level were the 
employees at before the program? In order to measure what the program has achieved, you need to 
be able to compare a pre-class assessment with a post-class assessment. So, in the sections that 
follow, keep in mind that the evaluation instruments will need to be used twice — before and after 
the program. It is essential that you use the same set of instruments in both assessments, so that 
you can compare them to see what your program has achieved. 

To give you some ideas for ways to evaluate your training program, this handbook includes a 
variety of evaluation instruments, which are described briefly below and explained in more detail in 
the rest of the handbook. For any one program, not all the instruments (or sections within an 
instrument) may be relevant. Some possible options are outlined here. 

1. As mentioned above in connection with Statistical Process Control, a very specific, job-related 
course may best be evaluated using only scenarios like those in the Employee Interview, in 
combination with employee job performance ratings. 

2. On the other hand, if longer-term and more general literacy gains are among a course's goals, 
then it makes sense Jto include the interview and questionnaire sections on learner practices, 
beliefs and plans. Gains in these areas are indicators of changes in lifestyle leading to greater 
flexibility and life-long learning. 

3. Another possible goal is increasing workers' ability to read workplace notices and newsletters. 
To assess changes in these skills, a prose scenario and a cloze exercise based on such 
workplace reading materials will most likely be appropriate. 

Other combinations will probably suggest themselves to anyone reading this with a particular 
workplace program in mind. For instance, your program may involve some ESL workers or place 
an emphasis on family literacy. There are instruments in this handbook for assessing learner gains 



in each of these cases. In addition, the handbook includes a classroom observation form, so that 
evaluators can record what happens during classes and relate teaching practices to assessment 
results. 



The nature of the evaluation instruments in the handbook is outlined below. Details are available in 
the introductory section for each instrument, which also includes guidelines for developing any 
custom-designed parts of the instrument. In the case of the Employee Interview, which contains 
open-ended questions with non-numerical responses, guidelines are also provided for conducting 
the interview and for scoring the responses. 



Employee Interview 

The interview form addresses learners' beliefs, practices, processes and plans relating to reading 
and education. It includes open-ended questions intended to find out about learner changes in 
practices, beliefs and plans, which can be used as they are by most programs. In order to find out 
about learners' reading processes, each program needs to develop one or more scenarios of the 
type included here, where interviewees are asked how they go about reading, as well as questions 
about the content of the material. 



Employee Questionnaire 

The questionnaire form addresses learners' literacy practices at work and away from work, and the 
literacy activities of their families. Most questions can be used as they stand, but some of the 
workplace reading materials should be chosen to be specific to the particular workplace. 



Cloze Exercise 

A cloze exercise is a fill-in-the-blank test that uses a passage of prase with (usually) eveiy fifth 
word removed. It tests a learner's ability to replace the missing words using the context of the 
passage— both its sense and its grammar. Cloze exercise scores can be related to standardized test 
scores, but have the advantage of being able to use a sample of the actual reading material required 
on th? job. Thus workers' ability to read job-specific prose can be assessed using a cloze exercise. 



Family Literacy Focus Group Interview 

One section of the Employee Questionnaire relates to family literacy, and this focus group 
interview form allows evaluators to follow up in more detail the types of responses obtained there. 



ESL Checklist 

For programs that include an ESL component, many of the other instruments may not be 
appropriate, particularly for beginning ESL learners who speak little English. Therefore, it may be 
more appropriate for the instructors to make their own assessment of the learners' abilities, and 
summarize that information on a checklist like the one given here. 
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Classroom Observation Form 



In order to be able to look at learner gains in the context of classroom practice, it is useful for an 
outside observer (possibly another teacher in the program) to watch some of an instructor's lessons 
in progress and make notes on what is happening. Then, if learners do (or do not) make gains in 
certain areas of the assessment, it may be possible to explain this in terms of classroom activities 
and emphases. 



Employee Job Performance Ratings 

In addition to direct testing of employee gains in skills and competencies, you will probably want 
to know how successfully they can apply their new knowledge back on the job. To find this out, 
the best way is to develop job performance rating scales in key areas, and ask supervisors (or 
others who are aware of individual performance) to rate the employees both before and after the 
training program. 



Employee Productivity Indicators 

Besides measuring improvement in job- specific skills, you may also want to look for indications of 
an increase in worker productivity, or improvements in other areas such as attendance and safety. 
For this, your organization probably already keeps the records you need, but you should check that 
they are available in a form that you can use to evaluate the employees in your program. 



Finally, remember that the instruments outlined above, and described in more detail below, provide 
a menu from which you can make a selection. And the most important factor in making such 
choices is the match between the instrument's purpose and your instructional goals: test what you 
are teaching and you are more likely to find out how successful that teaching has been. 



ERIC 



6 

7 



Employee Interview: Guidelines for 
Custom-Designing and Interview Form 



What modifications are needed? 

The Interview protocol that follows addresses learners' beliefs, practices, 
processes and abilities, and plans related to literacy activities. Most 
programs can use the supplied questions concerning beliefs, practices and 
plans without any modifications. 

For the process and ability section, job-specific modifications are required 
to determine how well employees read material from a particular workplace. 
You want to know how well employees use materials from your workplace, so 
you should select key items related to your own program's goals, such as: 

• a company newsletter, 

• a key graph or chart, 

• an instruction sheet, 

• a page from a manual. 

Using this material, develop two or three scenarios of realistic questions and 
tasks, to test the employees' understanding of how to use the material on 
the job. First, ask them how they would go about reading the material, to 
discover what reading processes they use. Then, in order to address a wide 
range of abilities, choose questions on the contents of the material that 
range from plain factual (the answer can be found in the material) to those 
requiring inference and application (the employee must make use of the 
information found). 

You will see from the examples given in the Interview below that a good way 
to achieve this is to include: 

• one process question — what is going on in the learner's mind 
(you can use the format in our examples), 

• two factual questions — based strictly on the material, 

• two inference questions — deductions from the material that 
do not rely on background knowledge, 

• two application questions — relating information from the 
material to the learner's background knowledge (usually the 
employee's job). 

Include a range of difficulty in these questions, finishing with an open-ended 
question which allows the interviewee to make a definite contribution. 
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Guidelines for Conducting Interviews 



The employee interview addresses employees 1 belief, practices, processes, 
and plans related to workplace activities. Appropriate skills for conducting 
interviews are required to elicit responses from the participants. Here are 
some points to keep in mind. 

Instructions 

• Read questions clearly to the employee. 

• Repeat questions if necessary. 

• Make notes on the responses in as much detail as possible. 

• Share what you have written to confirm responses. 



Prompting 

Prompting is used when employees stop after a single response. For 
example, while asking the employees the sorts of things they read and 
write, you can prompt them to give several examples. 

• Read the question and then the standard prompt(s) to all 
employees (e.g., "Can you give me more examples?"). 

• Avoid overguidance, such as suggesting possible responses. 

• Avoid personal interpretation and comments. 

• Keep a neutral tone. 



It is important to conduct the Interview in exactly the same way for both the 
pre-test and the post-test, so that any changes in scores can reasonably be 
attributed to the effect of your program. 




Guidelines for Scoring Interview Responses 



ERIC 



The employee interview will result in a set of interviewer's notes for each 
employee. In order to measure improvements produced by your program, 
you need some method of comparing the pre-test and post-test responses. 
Described below are scoring schemes that we have used for each of the 
areas: beliefs, practices, processes and plans. The following pages contain 
some examples of the use of these scoring schemes. 



Beliefs 

1. Rate the self-appraisal, if the learner has made one, on a scale of 1 - 5 
(i.e., 1 = poor, 3 = average, 5 - good, using 2 and 4 as needed). 

2. Make a holistic comparison* (see below) between pre- and post- 
responses. Make holistic judgments according to the following criteria: 
the reasons given for the reported self-image, in terms of abilities, 
activities and interests. 



Practices 

1. Count the number of items given in each response, and compare pre- 
and post-test results. 

2. Make a holistic comparison* (see below) between pre- and post- 
responses. Make holistic judgments according to the following criteria: 
breadth, frequency and difficulty of practices mentioned. 



Processes 

1. Count the number of items given in each response, and compare pre- 
and post-test results. 

2. To make a more detailed analysis of the responses, count the number of 
items which fall into the following categories: 

• points of focus— parts of the page (e.g., title, bold print, picture), 

• strategies— general reading strategies (e.g., skim, summarize), 

• topics— ideas from the material (e.g., parts, sales, scrap), 
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Plans 



For each part (1 year, 5 years and 10 years), make a holistic comparison* 
(see below) between pre- and post-responses. Make holistic judgments 
according to the following criteria: the clarity* definiteness and detail of 
the plans mentioned. 



* Making holistic comparisons 

• Read the pre-response and then the post-response. 

• Decide if the post-response is better or worse than, 
or about the same as, the pre-response. 

• Record changes as follows: 
M + ,f = better response 

" — " = worse response 

"0" = response about the same (use this one if in doubt). 
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Examples of Scoring for Interview Responses 



Beliefs 

2. How good do you consider yourself to be at reading and writing? 
What makes you think so? 

Pre: Not very good. Not much education. 
Post: I'm average. I'm not stupid. I have common sense and 
can read and write. 

Ratings of self-appraisal: pre is 2 for "not very good"; post is 3 for "average". 
Holistic comparison: + for increased valuing of abilities. 



Pre: Not so good at writing, but reading is OK. 

Post: Pretty good. Writing is difficult for me, but I can read OK. 

Ratings of self-appraisal: pre is 3 for "not so good" combined with "OK"; post is 

4 for "pretty good". 
Holistic comparison: 0 for no change in reasons for self-image. 



Practices 

2. Tell me the sorts of things you read and write on the job during a 
normal week. 

(For prompt ask: "Can you give me more examples?") 

Pre: Nothing really. I just put parts on the line. 
Post: Bulletin at work. I can really read it now. The information 
is important. I read the magazine at work also.. 

Counts of items: pre is 0; post is 2 for ''bulletin", "magazine". 
Holistic comparison: + for increased breadth, frequency and difficulty. 



Pre: Write work orders and read manuals all the time. 

Blueprints sometimes. 
Post: Manuals, memos, forms, work orders, computer screen. 

Counts of items: pre is 3 for "work orders", "manuals", "blueprints"; post is 5 
for "manuals", "memos", 'forms", "work orders", "computer screen". 
Holistic comparison: 0 for little change in nature of reading. 
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Process and Ability 



I am going to show you a newspaper article about your industry. I 
want you to explain to me how you would read it. There are many ways 
to read. People look at different parts of a page and think about 
different things when they read. 

{Show attached story: "Competitor Close Up"). 

1. (process question) 

What would you do first, then next, then next? 

Pre: Check each heading and decide whether to go further. 
Post: Read the headings, get ideas about the companies, skim, 
know what they make, and know their customers. 

Count for pre is 2: "check each heading" is a point of focus; "decide whether to 

go further" is a strategy. 
Count for post is 5: "read the headings" is a point of focus; "get ideas about the 

companies" is a topic; "skim" is a strategy; "know what they make" is a 

topic; "know their customers" is a topic. 

[N.B. Assigning a response to one of the three categories 'focus', 'strategy' 
and 'topic' is not always indisputable. For example, "check each heading"— 
described above as a point of focus— could also be interpreted as a reading 
strategy (i.e., skim the headings to find out what the article is about). 
Because of such ambiguities, it may be better to score these Process 
responses by counting only the total number of items, without making a 
break-down into categories.] 



Plans 

Now I'd like to ask you about your plans. Explain how you se e reading 
and education as part of these plans: 

1. What are your plans for the next year? 

Pre: Reading and writing more. Practicing. 

Post: I'll start college in September, to study accounting. 

Holistic comparison: + for increased definiteness and detail. 



Pre: Work in a better position. Work for people who need help. 
Post: f d like a better job, secretary maybe. 

Holistic comparison: 0 for no change in clarity, definiteness and detail. 
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EMPLOYEE INTERVIEW 
Personal Information: 

Name: Date: 

What class are you in? 

What job do you do? 



I'd like to ask you some questions about reading, writing, and 
education. The answers to these questions will give us an idea of the 
way reading and writing are used here. 



Beliefs 

1 . Describe someone you know who is good at reading and 
writing. What makes you choose this person? 



2. How good do you consider yourself to be at reading and writing? 
What makes you think so? 



3. Describe how you would li ke to be in terms of reading and 
writing. [Prompt : Could you give me some examples?) 



13 

14 



Practices 



1 . Tell me the sorts of things you read and write away from work 
during a normal week. 

(For prompt, ask: "Can you give me more examples?") 



2. Tell me the sorts of things you read and write on the job during a 
normal week. 

(For prompt, ask: "Can you give me more examples?") 
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Process and Ability. Article Example 



Competitor Close Up 

I am going to show you a newspaper article about your industry. I 
want you to explain to me how you would read it. There are many ways 
to read. People look at different parts of a page and think about 
different things when they read. 

[Show attached story: "Competitor Close Up"). 



1. (process question) 

What would you do first, then next, then next? 



(Thank the learner and read back what you have written to 
confirm the response.) 

Now go ahead and read the article. [Walt . . .) 

Now I am going to ask you some questions about the article. 

2. (easy factual question) 

How many employees does ASMO have in Statesville? 

(Answer: 400. Listed In article. ) 



3. (harder factual question) 

What is the only company that does not mention customers? 

(Answer: BG Automotive Motors, Inc. Requires the Interviewee 
to look at all "customers" In tae article. ) 



4. (easy inference question) 

From the information provided about products, what do all four 
companies have in common? 

(Answer: All of them make some sort of motor. Requires the 
Interviewee to search for commonalities not readily apparent. ) 
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Process and Ability : Article Example (cont.) 



5. (harder inference question) 

Which of the companies listed is closely related to Japan and 
why do you think so? 

(Answer: ASMO or Jideco. Each has Japanese plants listed 
and each sells to many Japanese affiliates and main 
customers. Requires looking at two pieces of information 
and drawing deductions based on what is provided. ) 



6. (harder application question) 

What company makes products closest to your job at this 
facility? Why do you say so? 

(Answer: Relate a product on the list to what the employee 
makes. Requires the employee to sort through the 
information and then to apply it to his/her background 
knowledge. ) 



7. (easy application question to end the section) 

From this list, which company pays the least amount to its , 
workers? How does this relate to your wages at Delco? 

(Answer: ASMO. It ! s more or it's less than what I get paid here. 
Requires the employee to apply the Information to his/her 
background knowledge, but allows him/her to contribute 
more. ) 
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Competitor Close-Up: 
A Year in Review 



Throughout the year, the Delco Doings 
has brought you profiles on the companies 
trying to take a bite out of our business and 
our profits. Sometimes there were success 
stories, when Rochester Operations met the 
challenge and came out on top. Other times 
we had to face the feet that there are 
companies in Asia, Europe, and right here 
at home that are reaching the market better, 
taster or with lower prices. 

Here's a quick recap of the competitors 
we've covered this year. 

ASMOj Inc. 

Location: Battle Creek, Michigan; Statesville, 
North Carolina; Kosai City, Japan. 
Affiliate: Nippondeso 

Products: wiper systems, windshield washer 
systems, power window lifts, antennas, 
retractable and blower motors. 
Main Customers: Nippondeso, Ford, 
Chrysler, General Motors, and every 
Japanese transplant except Nissan. 
Number of Employees: Battle Creek, 130; 
Statesville, 400. 

Total Wage and Benefit Cost/Hour. $9.58 
Jldeco 

Location: Bardstown, Kentucky;Yokohama 
City, Japan; Nine production facilities 
throughout Japan 

Affiliates: Hitachi (24%), Nissan (21%) 
Products: wipers, transmissions, reservoirs 
arms and blades, wiper motors, and others. 
Control-- wiper switches and others. 
Motors -- power seat sliders, power window, 
door lock, blower and engine cooling 
motors and others. 

Accessories: air compressors, power window 

kits, door locks, rain-sensing intermittent 

wiper controls and others. 

Main Customers: Nissan, Isuzu, Honda, 

Mitsubishi, Mazda, and Suzuki. 

Number of Employees: Bardstown, 60 in 1987 

Total Wage and Benefit Cost/Hour: $ 10.27 



Power Motion 

Location: Two plants in London, 
Ontario 

Parent: Siemans Automotive of 
West Germany 

Products: air moving motors (5,250 
armatures a day) 

Main Customer: GM of US & Canada 
Number of Employees: 200 at main 
facility in London, Ontario 
Total Wage and Benefit Cost /Hour: 

$11.50 (U.S. equivalent) 

BG Automotive Motors. Inc. 

Location: Hendersonville, TN. 

Parents: Bosch Corporation and General 

Electric Company 

Products: 20 different small motors 

including: engine cooling, modular wipers, 

door lock, seat back, head rest, sun-roof, 

washer pump, head lamp, power window. 

Number of Employees: 275 

Total Wage and Benefit Cost /Hour. 

Unknown at this time. 

Every day another company steps into 
the automotive arena ready to try to take 
away our customers. Rochester Operations 
has an extensive communication network 
to keep employees informed about our 
competitors and what we're doing to stay 
ahead. Look to Delco Doings to give you 
the information you need to help keep 
Rochester Operations competitive in the 
•90s. 



From: Delco Doings, December/ January, 1991, p.2. 
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Process and Ability. Graph Example 



Production Problems 



I am going to show you a graph. I want you to explain to me howyou 
would read it. There are many ways to read. People look at different 
parts of a page and think about different things when they read. 
(Show attached graph." Production Problems ). 

1. (process question) 

What would vou do first, then next, then next/ 



(Thank the learner and read back what you have written U 
confirm the response.) 

Now go ahead and read the article. (Walt . . .) 

Now I am going to ask you some questions about the graph. 

2. (easy factual question) 

What is the total number of culls? 

(Answer: 149. Shovm at top of graph.) 



3. (harder factual question) 

What time period is covered in this chart? 

(Answer: one week or week one in May. S. own at top q 
graph in abbreviated form.) 



4. (easy inference) 

What is the biggest problem here? 

(Answer: tear outs. Longest bar on graph.) 
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Process and Ability : Graph Example (cont.) 

5. (harder inference) 

Find 3 types of problem involving measurement. 

(Possible answers: thickness, length, width, squareness. 
Requires selection from list at left of graph.) 



6. (easy application question) 

Pick one problem and suggest at least one cause for that 
problem. 

(Possible answers: For example, tear outs are caused when 
the wood gets caught in the machine and is gouged; moulder 
burn is caused by wood getting caught in the machine and 
being burned. Uses interviewee's job-related knowledge. ) 



7. (more difficult application question) 

Pick a second problem and suggest both a cause and a solution 
for the problem. 

(Possible answers: tear outs, caused when the wood gets 
caught in the machine and is gouged, can be repaired with 
wood filler and sanding; or moulder knife marks can be 
caused by gouging of the wood in carving it and can be 
repaired if you can get at the gouge and sand it and 
provided the finish hasn't already been applied. Uses 
interviewee- s job-related knowledge in more depth.) 
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Process and Ability: Procedure/ Job Aid Example 



OSHA CARD 

The government has safety regulations and special labels in many 
workplaces. I am going to shSw you a safety card that many employees 
in America must keep in their pockets while working. This card 
shows how to understand safety labels. 

I want you to explain to me how you would read it. There aiemany 
ways to read. People look at different parts of a page and think about 
different things when they read. 

[Show attached card , "OSHA "). 

1. (process question) 

What would you do first, then next, then next? 



[Thank the learner and read back what you have written to 
confirm the response.) 

Now go ahead and read the article. [Wait . . . ) 

Now I am going to ask you some questions about the card. 

2 (easy factual question) 

What should you do when you see the letter x / 

(Answer: Ask my supervisor. Directly explained in the text ) 



3. (harder factual question) 

What do all the symbols in u k M represent? 

(Answer: airline hood or mask, gloves, a suit and 
Answers are in the text, but are more difficult to 



(easy inference) » v « 0 
What is the most common type of protection from A to iw 

(Answer: gloves. Requires the interviewee to look through 

several parts of the text and then to generalize the injormatton ) 



Process and Ability: Procedure /Job Aid Example (cont.) 



5. (harder inference) 

Name all the letters which refer to severe hazards. How did 
you tell this? 

(Answer: F, H, J. K. Top of the table says "4 severe hazard; 
4 probably means 4 pictures. These letters have 4 pictures. 
Requires the interviewee to make deductions between 
different parts of the card. ) . 



6. (harder application) 

If a supervisor says you are about to do a job that requires sanding, 
which protective items would you choose? 

(Answer: safety glasses and a dust respirator. Optional: gloves, 
combination dust /vapor respirator and a face shield. Requires 
the interviewee to interpret the information on the card and 
to relate it to a real-life situation. ) 



7. (easy application question to end the section) 

Give me two examples of how you or someone you know 
could use this card. 

(Answer: Must give 2 examples and list protections. This 
is more open-ended and allows the interviewee to contribute 
based on his/her job background.. ) 
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Plans 



Now I'd like to ask you about your plans. Explain how you 
and educat ion as part of these plans: 

1 . What are your plans for the next year? 



2. What are your plans for the next 5 Years? 



3. What are your plans for the next 10 Years? 
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Employee Questionnaire: Guidelines for 
Custom-Designing and Questionnaire Form 

What modifications are needed? 

The Questionnaire protocol that follows addresses learners 1 literacy 
practices at work and away from work, and the literacy activities of their 
families. Most programs can use the supplied questions concerning: 

• practices at work 

• practices away from work 

• family 

with few modifications. 

In the section on practices at work, the majority of questions should apply 
to most industries and thus need no changes. However, items 7-10 (job 
order form, route sheet, computer screen, blueprint) may not apply to your 
workplace and should be changed to site-specific reading materials, such as 
warning labels, procedures, product lists, etc. Actual names may differ from 
site to site. 

When you choose these last 4 items, select a mix of: 

• prose and graphic materials (e.g., a note from a 
supervisor, and a blueprint) 

• easy and difficult reading materials (e.g., simple 
suggestion forms and more complex benefit 
information) 
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INSTRUMENT EXAMPLE: LEARNER QUESTIONNAIRE 

Name: Date: 

Education: (last year completed) Training: 

Age: Sex: Children: (number) (ages) 

Questionnaire: Practices at work 

Please check the number of times you have done the following: 

1. In the last 7 days how many times have you read a company 

newsletter or bulletin board? 

0 1 2 3 4 5 6~ 7 8 9 10+ 

2. In the last 7 days how many times have you used a manual? 

0 1 2 3 4 5 6 7 8 9~ ?0+ 

3. In the last 7 days how many times have you read a memo or letter? 

0 ~1 2 3~ 4 5 6 7 8 ~9~~ 10+ 

4. In the last 7 days how many times have you used a graph or chart? 

0 1 2 3 4 5 6 7 8 9 10+ 

5. In the last 7 days how many times have you used an instruction sheet? 

0 1 2 3 4~ 5 6 7 8 9 T5+ 
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Questionnaire: Practices at work (cont.) 

6. In the last 7 days how many times have you written a note to 

a co-worker? 

0 1 2 T 4 5~ 6~ 7 8 9 T5+ 

7. In the last 7 days how many times have you used a job order form ? 

0 1 2 T 4 5~ 6 ~7~ 8 9 76+ 

8. In the last 7 days how many times have you used a route sheet ? 

0 1 2 ~3~~~4 5~ 6~ ~7 8 9 10+ 

9. In the last 7 days how many times have you used a computer screen 

0 1 2 3~ 4 5~ 6 T~ 8 9 10+ 

1 0. In the last 7 days how many times have you used a blueprint ? 

0 1 2 T 4 5" 6 7 8 9 10+ 
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Questionnaire: Practices at work (cont.) 



11. How often do you have difficulties with the paperwork in your job? 
never occasionally often always 

1 2. How often do you wait for others to talk about written information, 

just to be sure what is in it? 

never occasionally _„_ often always 

13. How often do you have trouble reading paperwork from management? 
never occasionally often always 

14. When a problem arises, how often do you look for printed directions 

to help figure out what to do? 

never occasionally often always 

15. How often are you able to help when someone has trouble 

reading something? 

never occasionally often always 

16. How often do you talk in team or department meetings, asking 

questions or sharing ideas? 

never occasionally often always 

1 7. How often are your ideas discussed in team or department 
meetings? 

never occasionally often always 

28 



Questionnaire: Practices away from work 

Please check the number of times you have done the following: 

1 . In the last 7 days how many times have you read a newspaper? 

0 1 2 T 4 5~ 6 7 8 9 10+ 

2. In the last 7 days how many times have you read mail, bills or ads? 

0 1 2 3~ 4 5 6 7 8 9 10+ 

3. In the last 7 days how many times have you read a magazine? 

0 1 2 3 4~~ 5 6 7 8 9 T5+ 

4. In the last 7 days how many times have you read a book for pleasure? 

0 1 2 3 4 5 67 8 9 10+- 

5. In the last 7 days how many times have you read in order to do 

something? (For example: buy, build, cook, fix) 

0 1 2 3 4 5 6 7 8 9 10+ 

6. In the last 7 days how many times have you discussed something 

you've read with another person? 

0 1 2 3~ 4 5 ~6 7 8 9 10+ 

7. In the last 7 days how many times have you written notes to people 

you live with? 

0 1 2 3 4~~ 5 6 7 8 9 T5+ 
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Questionnaire: Practices away from work (cont.) 



8. In the last 7 days how many books have you brought into your home, 

either bought or borrowed? 

0 1 2 3 4 5~~ ~6 7 8 9 T6+ 

9. In the last 7 days how many times have you read the following 
types of books ? 

mystery: times how-to books: times 

novels: times factual books: — times 

poetry: times encyclopedia: — times 

Bible: times comic books: times 

other types: — times 



1 0. When you're waiting in an office, how often do you read magazines? 



1 1 . How often do you make a shopping list before you go to the store? 



times 



times 



never 



occasionally often 



always 



never 



occasionally 



often 



always 
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Questionnaire: Family 

Only answer the following questions if you have a child 
between the ages of 3-17 at home. 

Please answer for your youngest child in this age group and please 
fill in only one answer per question: 

1 . This child is years old. 

2. In the last 7 days how many times has your child looked at or 

read books or magazines? 

0 1 2 3 4 5~ 6 7 8 ~9~ 76+ 

3. In the last 7 days how many times has your child seen you 

reading or writing? 

0 1 2 3 4 5~ 6 ~7 8 9 10+ 

4. In the last 7 days how many times have you helped your child 

with homework and/or with school projects? 

0 1 2 3~ 4 5 6 7~~ 8 9 10+ 

5. In the last 7 days how many times have you read or looked at 

books with your child or listened to him/her read? 

0 1 2 3 4 5 ~6~~ 7 8 9 ?0+ 

6. In the last 7 days how many times has your child asked to be 

read to? 

0 1 2 3~ 4 5 6 T 8 ~9~ T5+ 

7. In the last 7 days how many times has your child printed, made 

letters, or written? 

0 1 2 3 4 5 6 7 8 9 T5+ 
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Questionnaire: Family (cont.) 

8. In the last month how many times has your child gone to a 

public library? 

0 1 2 3~ 4 5~ 6~~ 7 8 9 ~~To+ 

9. In the last month how many times have you participated/helped 

out in your child's school? 

0 1 2 3~ 4 5 6 7 8 9 10+ 

1 0. In the last month how many times have you hung up or displayed 

your child's reading and writing efforts? 

0 1 2 ~3~ 4 5 6 7 8 9 10+ 

11. In the last month how many times have you bought or borrowed 

books for your child? 

0 1 2 3 4 5~ 6 7 8 9 To+ 

12. (Please circle only one.) 

1 expect my child to finish at least: 

6th grade 9th grade high school 2-year college 4-year college 
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Cloze Exercise: Guidelines for 
Custom-Designing and Example Tests 



The cloze procedure is based on the psychological principle of 
closure, which is the human tendency to recognize and complete a 
pattern or sequence. It involves replacing missing words in a reading 
passage. This procedure can assess the ability of employees to 
comprehend the passage. Cloze tost scores correlate very highly with 
standardized reading test scores. Cloze tests can be made from local 
workplace materials in the following way. 



1. Select a job relevant passage of 150-200 words. 

2. Leave the first and last sentences intact. 

3. Starting with the second sentence, omit every fifth 
word. This will give about 25 blanks. Replace all 
omitted words with the same sized blank line, 
13-15 spaces is typical. 

4. Employees are to read the passage and to fill the 
blanks with their best guess at the word removed. 

5. Instructions for the cloze test should suggest 

employees read the entire text before attempting 
to fill in the blanks and should encourage 
employees to answer all questions even if they 
have to guess. It is rare for anyone to know more 
than 50% of the blanks. 

6. Avoid controversial or emotional topics, and 

topics requiring technical knowledge. Scores for 
such materials are less valid. 
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Name Date 

CLOZE Exercise 

In a cloze exercise, you try to guess which words are missing. For 
example, in the sentence below, a word is missing. 

She looked before she the street. 

A good guess for the missing word is "crossed." 

She looked before she CTOSStd the street. 

In the story below, try to guess and replace the missing words. Don't 
expect to get them all. Some are nearly impossible. 

G.M Designs Safety for All Ages 

We all like to think about the old days. Life seemed simpler and, in 

some ways, better then. But when it comes to _, the 

good old days offer the same degree 

safety as today's cars trucks. Advancements in 

technology the G.M. vehicle you 



today among the safest the world. Each G.M. 

and truck is backed thousands of 

dedicated men women who care about 

safety of their customers. , as G.M. 

customers themselves, have a stake in 

G.M. vehicles the highest quality 

and reliability. 

And you're wondering if safety 
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improved in recent years, this: The classic 1955 

would require more than major 

changes or additions hundreds of incremental 

changes be as safe as vehicles. 

From: Kilborn, C. GM Today (November/December, 1990), page 1. 
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Cloze Exercise Answer Key 
G.M Designs Safety for All Ages 

We all like to think about the old days. Life seemed simpler and, in 
some ways, better then. But when it comes to ailtomoBifes , the 
good old days didn't offer the same degree of safety 
as today's cars and trucks. Advancements in 
technology make the G.M. vehicle you purchase 

today among the safest in the world. Each G.M. 

COT and truck is backed By thousands of 

dedicated men and women who care about 

the safety of their customers. ftnd , as G.M. 
customers themselves, the.}} have a stake in 

making G.M. vehicles the highest in quality and 
reliability. 

And if you're wondering if safety flOS 
improved in recent years, consider this: The classic 1955 

Chewotet would require more than 60 major 

changes or additions and hundreds of incremental 

changes to be as safe as today's vehicles. 
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Name or ID# Date 

CLOZE Exercise 

In a cloze exercise, you try to guess which words are missing. For 
example, in the sentence below, a word is missing. 

She looked before she the street. 

A good guess for the missing word is "crossed." 

She looked before she crossed the street. 

In the story below, try to guess and replace the missing words. Don't 
expect to get them all. Some are nearly impossible. 

Two more teams on the se lf-directed iournev 

Our workplace is taking on more change daily. So are the skills 
that all our employees must have in order to change with it. It is 

getting to be essential, each day that skill 

gaps be filled our small business can a 

source of competitive • 

The changes all companies expect over the next 

a shrinking labor force, demand for 



workers in jobs, and increasingly competitive 

markets -- will require businesses all sizes 

to strengthen employee skills and training 



We believe that our firm can remain competitive 

the large firm by a more flexible training 

education program. We hope be better than the 

firm in adapting an previous training 



experiences to company's needs. 

Two more are now involved in self-directed 

team training. They are Green Team and the 

Team. They join the Orange Team, which completed their sessions 
last year. 
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Cloze Exercise Answer Key 
Two more teams on the self-directed journey 

Our workplace is talking on more change daily. So are the skills 
that all our employees must have in order to change with it. It is 

getting to be more essential each day that the 

skill gaps be filled so our small business can remain 

a source of competitive strength . 

The changes all companies can expecl over the next 
decade a shrinking labor force, more demand for workers 
in technical jobs, and increasingly competitive world markets 
- will require businesses of all sizes to strengthen their 
employee skills and training programs . 

We believe that our small firm can remain competitive with 
the large firm by having a more flexible training and 
education program. We hope to be better than the large 
firm in adapting an employee's previous training experiences to 
the company's needs. 

Two more teams are now involved in self-directed work 
team training. They are the Green Team and the U/hite 
Team. They join the Orange Team, which completed their sessions 
last year. 



ERLC 



37 

4 0 



Name or ID# 



Date 



CLOZE Exercise 

In a cloze exercise, you try to guess which words are missing. For 
example, in the sentence below, a word is missing. 

She looked before she the street. 

A good guess for the missing word is "crossed." 

She looked before she crossed the street. 

In the writing below, try to guess and replace the missing words. 
Don't expect to get them all. Some are nearly impossible. 

Safety Rules 

1. For your welfare, all injuries, no matter how slight, incurred on Company premises 
must be reported immediately to your supervisor. The services of a physician 

available and will be as required. Failure to 

such injuries may cause difficulties and could affect 



Workingmen's Compensation benefits. 

2. Wear and shoes suitable to 

top shoes are not 



work. 



Open toed or 

are required. 

3. Dust your only with an air 



. Shorts are not permitted. 



equipped with an air 



its proper place and 



.__ nozzle. 

4. Keep fire equipment 

rules. 

5. Learn lift properly. Keep your 

your to avoid strain. 

6. All guards should be kept 



all fire 



straight and use 



machine guards be reported to your 

guard be removed without the 

7. Do repair machinery when It 



place. Unsafe 
at once. No 



of your supervisor. 
in operation. 



Stop it and fix the switch so that it cannot be accidently turned on. 
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Cloze Exercise Answer Key 
Safety Rules 



1 . For your welfare, all injuries, no matter how slight, incurred on Company premises 
must be reported immediately to your supervisor. The services of a physician 

are available and will be obtained as required. Failure to 
report such injuries may cause medical difficulties and could affect 

uour Workingmen's Compensation benefits. 

2. Wear clothing and shoes suitable to your work. 

Open toed or canvas top shoes are no t permitted . Shorts are not permitted. 

Shirts are required. 

3. Dust your clothes o.Jy with an air _hose equipped with an air 

restricting nozzle. 

4. Keep fire equipment in is proper place and obey all fire 
rules. 

5. Learn to lift properly. Keep your back straight and use 
you r leas to avoid strain. 

6. All machine guards should be kept in place. Unsafe 

machine guards should be reported to your supervisor at once. No 
guard should be removed without the permission of your supervisor. 

7. Do not repair machinery when it is in operation. 

Stop it and fix the switch so that it cannot be accidentally turned on. 
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Family Literacy Focus Group Interview 



This interview form is designed to be used with a group of 
learners as the basis for a discussion about family literacy. It has been 
found that the comments of one member of the group will stimulate 
the thoughts of others, producing a wider range of ideas than will 
individual interviews. 



40 13 



Family Literacy Focus Group Interview 



Why do you think some children learn to read and write well 
in school and others don't? 



What do you think parents can do to help their children learn to 
read and write better? 



What kinds of reading or writing materials do you keep at home 
for your children? 

(i.e., letter blocks, flashcards, paper, pens, chalkboard, books, 
magazines, comics, cassettes with books, encyclopedia, 
dictionary, newspapers, etc) 
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What reading or writing activities do you do with your children? 
(i.e., visit library, hear stories, read to them, watch educational 
television, look at magazines or books with children, point out 
words to them, play school, show them how to read or write, et 



At home, what reading or writing do your children see you doing? 
(i.e. books, magazines, papers, recipes, directions, letters, lists, 
notes, etc.) 



What activities are you involved in at your child's school? 
(i e , parent/ teacher meetings, school fund-raisers, 
committees, notes or letters, informal talks when collecting 
child, assist in classroom, help child read at home, etc) 
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7. Have you begun anything new related to reading and 
writing since you started classes here? 



a. Materials 



b. Activities 



c. Modeling 



d. School 
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ESL Checklist 



The ESL checklist is designed for teachers to reflect upon the level 
of competence each student is demonstrating. Teachers will be able 
to note individual areas of strength and weakness. This form is helpful 
both in planning instruction and in suggesting areas for the student to 
practice on outside of the workplace. 




ESL Benchmarks and Ratings* 
Learner Name Teacher Name 



Date of Rating 
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For each item, rate the learner: 

3 = can do this as well or nearly as well as a native speaker 
2 = can usually manage to do this, but sometimes has trouble 
1 = can only sometimes manage to do this adequately 
0 = cannot do this 

Beginner Level 

Briefly describes feelings about work 

Briefly describes feelings about other life areas 

Follows simple directions 

Asks for clarification if something is not understood 

Reads alphabet in English 

Word recognition: 

has access to dictionary /understands dictionary use 

uses dictionary 

uses roots, prefix, suffix 

uses context 

Looks up simple information (phone book, dictionary) 

Reads simple signs 

Begins short journal entries 



* Modified from Bronstein, E. (1991) Benchmarks and student learning profile for the 
workplace ESL program of the Labor Education Center at Southeastern Mass. University 
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Intermediate Level 
Oral 

Discusses feelings about work with some elaboration 

Discusses feelings about other life areas with some elaboration 

Gives /follows instructions at work 

Gives /follows instructions in other life areas 

Asks for clarification if something is not understood 

Discusses industrial specific diseases /illnesses 

Describes /reports dangerous conditions 

Offers suggestions to supervisor 



Reading 

Uses dictionary ( bilingual English-English) 

Locates own reading material in newspapers 

Understands literal level of text 

Infers information not explicitly stated 

Draws conclusions from reading 

Writing 

Fills out more complex forms 

job application social security insurance 

other application forms (library card, courtesy card, credit 

card) 

Writes short notes /memos ( at work 

out of work) 

Writes journal entries (dialogue journal) 

Uses correct punctuation 
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Advanced Level 
Oral 

Discusses feelings with more elaboration 

Asks for clarification if something is not understood 

Gives /follows more complex directions 

Understands and can discuss basic worker rights 

Understands and can discuss key contract sections 

Reading 

Uses index/ table of contents 

Locates own reading material in newspaper 

Locates own reading material in encyclopedia or other 

reference 

Reads short pieces in newspaper and simple magazines 

Reads short pieces in flyers, notices, factsheets 

Reads short self-selected material at home 

Reads longer pieces in books or longer articles 

Writing 

Fills out more complex forms 

job application social security insurance 

other application forms (library card, courtesy card, credit 

card) 

Writes short notes /memos ( at work 

out of work) 

Observes differences in tone/register between formal and 

informal writing 

Writes longer journal entries /responds to entries 
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Classroom Observation Form 



The classroom observation form serves as a guide for recording 
notes about the activities actually occurring in the classroom. It is 
divided into columns reflecting the time in five minute intervals, the 
actual activities of both teacher and student, and comments about the 
overall class. The form suggests items the observer might wish to 
note. 
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Classroom Observation 



Time 

0 



Student Activity 



Teacher Activity 



Comments 



10 
15 



20 
25 



30 
35 

40 



45 



50 
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Make note of time spent by students actually reading or doing things. Also note time students 
spend listening to the teacher, working in small groups, or working individually. Special note 
should be made when teacher or student demonstrates how to do something. 
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Employee Job Performance Ratings: Guidelines 
for Custom-Designing and Example Scales 



When all learners In a class have similar jobs or have the same 
supervisor or team leader, a set of shared job performance ratings can be 
developed to measure productivity. Supervisors or team leaders often have 
direct knowledge of employees' job performance, so it is best to develop 
these ratings with them. To assess the impact of training on the employees, 
these ratings should be used as pre- and post-indicators. 

1. Ask about use of job skills. 

Ask supervisors or team leaders who have employees in training to 
describe how top performers use information on the job. Encourage 
them to think of specific workers who are top performers. One might 
say for example, that a top performer reads charts and responds with 
his own analysis, or sets machines correctly and checks settings 
thoroughly, or completes all job-related paperwork and tries to 
improve procedures. Continue to probe until you feel reasonably 
satisfied you have a complete list from the supervisors or team leaders. 
From this list, you can identify important areas (i.e., communication, 
problem solving, paperwork, etc.) 

Next go through a three-step process in fleshing out these areas. The order 
of these steps is important. 

2. Obtain descriptions of performance. 

Ask supervisors or team leaders to: 

a. describe the behavior of the top performers first; 

b. then, describe the behavior of the bottom performers; 

c. last, describe the average performers. 

These behaviors will be used to provide descriptions and anchors for 
ratings. In relation to paperwork, for example, supervisors or team 
leaders might agree on the following descriptions: 

Top: completes all job-related paperwork and tries to improve 
procedures; 

Bottom: intimidated by job-related paperwork and does it poorly; 
Average: does job-related paperwork but simply keeps pace. 
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As you work to develop these descriptions, new areas and categories 
may emerge. The supervisors or team leaders may give examples 
related to problem-solving or to machine setting, or some other area. 
These may later become additional rating scales. 



3. Develop labels for performance areas. 

Once the descriptions of top, bottom, and average performances are 
completed, work with supervisors or team leaders to develop 
acceptable labels for the categories. For example, labels might include 
items like machine setting, paperwork, communication, 
responsibility, and problem-solving. 



4. Draft a rating scale. 

After this discussion, draft a rating scale and submit it to the people 
you interviewed for comment and possible revision. Sometimes during 
revision, complex scales split to become two separate scales. 



Examples of scales appear on the following pages. 



Employee Assessment - Overall Rating 



Please rate each employee on a scale of 1 - 10 for each aspect below. 

• An average employee would be rated 5. 

• A top employee would be rated 8 or higher. 

• A bottom employee would be rated 2 or lower. 



EMPLOYEE 



DATE 



RATER 



COMMUNICATION 
Bottom 

doesn't talk about the job, 
doesn't speak at department 
meetings 



Average 

talks about the job and 
offers suggestions at 
department meetings 



Toe 

talks about the job, has 
suggestions and shows 
leadership 



4 



7 8 



10 



PROBLEM-SOLVING 
Bottom 

calls supervisor on minor 
details or continues to work 
when equipment is faulty 



Average 

makes minor adjustments, 
offers solutions to problems 
and calls supervisor only 
when necessary 



7 8 



Toe 

can analyze job situations, 
make suggestions and 
solutions which implement 
change 



10 



QUALITY OF PAPERWORK 

Bottom Average 



paperwork provides no 
or limited information; 
illegible, poor grammar 



paperwork usually 
acceptable; at times too 
brief or vague 

5~"~6""~7"~8 



Top 

paperwork is legible, 
detailed, clear and 
concise 



10 
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SAFETY PRECAUTIONS 



Bottom 



Average 



Top 



has to be reminded about 
safety and does not use 
caution on the job 



takes precautions, but 
sometimes overlooks 
small areas on the job 



follows safety precautions 
and understands why 



7 8 



10 



PROCEDURES 
Bottom 

does not follow procedure 
in a step-by -step process, 
sometimes missing areas 
of the job 



Average 

follows procedure most 
of the time, occasionally 
missing a step 



Top 

follows step-by-step 
procedure 



7 8 



10 



MACHINE SETTING 
Bottom 

unable to set 
machines correctly 



Average 

usually sets 
machines correctly, 
but doesn't always 
check settings 



Top 

sets machines 
correctly and 
checks settings 
thoroughly 



7 8 



10 
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Informal Employee Job Assessment: 
Focus Group Interview of Supervisors 



If there is not enough time or opportunity to develop formal rating scales in 
the manner described above, then an alternative which will give some idea 
of the gains produced by the training program is to conduct informal 
interviews with groups of supervisors or team leaders, as described below. 

At the end of the training, the immediate supervisors or team leaders of the 
employees in the training class are interviewed as a group (of perhaps 4 or 
5), in order to find out what changes they have noticed in the workplace 
behaviors of those employees who have just been trained. Questions could 
include: 

1 . What effects of the training class have you seen in the workers under your 
supervision or in your team? 

(Answers might include: more cooperative, work better in the team, 
take greater responsibility, improved self-confidence.) 

2. What can they do now, but could not do before? 

(Answers might include: set machines accurately, check for machine 
errors, complete dally log, do routine work without close supervision.) 

3. Has anyone else noticed that? 

(Follow up responses to Questions 1 and 2 to find out how many 
workers have changed in these ways.) 
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Employee productivity indicators 



Besides measuring improvement in job-specific skills, you may also want to 
look for indications of an increase in worker productivity, or improvements 
in other areas such as attendance and safety. For this, your organization 
probably already keeps the records you need, but you should check that they 
are available in a form that you can use to evaluate the employees in your 
program. For example, production quality data can measure the success of 
the training program only if pre- and post-data for those employees in 
training can be isolated from that of other employees. 

This section discusses several kinds of organizational records which you 
could use as pre- and post-measures to evaluate the impact of a workplace 
training program. Records include: 

• Productivity indicators 

• Attendance and punctuality records 

• Safety records 

• Productivity suggestions 

• Retention and promotion records 

• Records of improvement in plant-specific goals (i.e., increased 
participation in quality assurance activities, increased enrollment in 
voluntary education, etc.) 

Some of these indicators occur infrequently (e.g., accidents, breakdowns, 
promotions). For this reason, these indicators are most useful when there 
are large numbers of employees in the evaluation or when pre /post 
comparisons cover long periods of time. 



Productivity indicators 

Many employers already collect data on productivity (i.e., defects, parts 
produced, customers serviced, complaints, etc). To be effective, 
information must be collected about each individual worker so changes 
resulting from that employee's training can be noticed. You should compile 
a list of productivity data currently gathered by your company. Team 
productivity data can be of use only when entire teams train together. 
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Attendance & punctuality 



Sometimes training makes people want to come to work more often. Some 
programs have been able to demonstrate a positive impact on work 
attendance as a result. For this reason, it is useful to gather attendance 
information for both classes and work. Because absenteeism is usually 
infrequent, however, program impact can be influenced greatly by a single 
individual with many absences or by legitimate absences which influence 
everybody (i.e., health during flu season). For this reason, interpretation of 
attendance data must be done very carefully. Similar care must be taken 
with punctuality data — especially when only a small number of employees is 
involved. 



Safety 

In most industries, safety is a major concern for both employers and 
employees. Training classes often address safety issues and some programs 
directly support these efforts. It is difficult to demonstrate impact on safety, 
however, unless one collects information on large numbers of workers for a 
relatively long period of time before and after training. 



Su ggestion Forms 

Most companies have suggestion boxes for collecting employee productivity 
and safety ideas. Employees are often more willing to express their opinions 
as they become more competent with reading and writing, and have 
experience writing suggestions forms. Some businesses also keep track of 
the dollar value of accepted suggestions. 



Retention & promotion 

Workplace training programs directly designed to support promotion or 
retention of employees during times of job restructuring are most likely to 
demonstrate impact. To determine impact, it is important to compare 
learners receiving training to comparable groups who did not receive 
training. In addition many companies have tests to determine promotion 
qualification. Comparing scores of program participants to other test takers 
or to the participants' own earlier test scores can assess impact on 
promotion. Similar patterns are possible in assessing who is retained during 
employer downsizing. 
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